Optimal Bandwidth Allocation With Delayed State Observation and Batch Assignment

In this paper we consider the problem of allocating bandwidth/server to multiple user transmitters/queues with identical but arbitrary arrival processes, to minimize the total expected holding cost of backlogged packets in the system over a finite horizon. The special features of this problem are that (1) packets continuously arrive at these queues regardless of the allocation decision, (2) there is a large delay between when the allocation decision is made and when the allocation is used, thus decisions can be viewed as based on delayed or obsolete state information/observations; and (3) the bandwidth allocation is in batches of time slots so that each queue can be assigned any number of slots not exceeding the total number in a batch. This problem is motivated by channel allocation in a communication system involving large propagation delay, e.g., a typical satellite data communication scenario. This problem can be cast as a special case of the restless bandit problem with multiple plays with the additional properties that the state observation is imperfect and that each queue can be served multiple times in between two decision epochs. We identify two sufficient conditions which jointly define a class of policies that are optimal. We further show that under very mild assumptions on the arrival process these two conditions are also necessary for the optimality of a policy. Without these assumptions on the arrival process the two conditions are not in general necessary, which we show via two examples. Index Terms Resouce allocation, multi-armed bandit, restless bandit, optimal policy